allow restricting Kube metadata to local node only #1440
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1440 +/- ##
==========================================
+ Coverage 80.97% 81.14% +0.17%
==========================================
Files 149 149
Lines 15255 15309 +54
==========================================
+ Hits 12353 12423 +70
+ Misses 2293 2274 -19
- Partials 609 612 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
mariomac
requested review from
grcevski,
marctc and
rafaelroquetto
as code owners
|
Hello @mariomac!
Please, if the current pull request addresses a bug fix, label it with the |
|
This PR must be merged before a backport PR will be created. |
There was a problem hiding this comment.
LGTM! I think the test failure might be related to aggressive expiration of metrics, or maybe confusion of what the name should be. I see an expiration of these values, where we have testserver-....
2024-12-12T16:40:35.912684579Z stdout F time=2024-12-12T16:40:35.912Z level=DEBUG msg="storing new metric label set" component=otel.Expirer type=*metric.int64Inst labelValues="[172.18.0.2 43558 request 10.244.0.5 10.244.0.0/16 testserver-858cdf668b-xpnlx 8080 egress my-kube testserver-858cdf668b-xpnlx default 172.18.0.2 test-kind-cluster-netolly-control-plane testserver Deployment Pod internal-pinger-net default 172.18.0.2 test-kind-cluster-netolly-control-plane internal-pinger-net Pod Pod 8080 10.244.0.9 10.244.0.0/16 internal-pinger-net TCP]"
However the next label is recorded with testserver as a name, i.e. no dash something:
2024-12-12T16:41:02.912577238Z stdout F time=2024-12-12T16:41:02.912Z level=DEBUG msg="storing new metric label set" component=otel.Expirer type=*metric.int64Inst labelValues="[172.18.0.2 43558 request 10.96.94.38 10.96.0.0/16 testserver 8080 egress my-kube testserver default testserver Service Service internal-pinger-net default 172.18.0.2 test-kind-cluster-netolly-control-plane internal-pinger-net Pod Pod 8080 10.244.0.9 10.244.0.0/16 internal-pinger-net TCP]"
All subsequent labels are without the dash...
pkg/kubecache/meta/informers_init.go
Outdated
| @@ -470,6 +521,10 @@ func (inf *Informers) ipInfoEventHandler(ctx context.Context) *cache.ResourceEve | |||
| return &cache.ResourceEventHandlerFuncs{ | |||
| AddFunc: func(obj interface{}) { | |||
| metrics.InformerNew() | |||
| em := obj.(*indexableEntity).EncodedMeta | |||
| for _, ip := range em.Ips { | |||
There was a problem hiding this comment.
I think we probably want to remove this debug print :)
|
Good catch @grcevski ! The duplicity of messages might be because testservice is captured either via a Pod (name with suffix) and a Service (name without suffix), but I'll anyway check that the IPs do not collide and that the expiration is properly set. EDIT: you were right! But the expiration was not too early in Beyla, as I initually understood, but in the Prometheus TSDB. |
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-1440-to-release-1.9 origin/release-1.9
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 3b31c2e0be030243e4f8ca73a61fcefbb0387ecdWhen the conflicts are resolved, stage and commit the changes: If you have the GitHub CLI installed: # Push the branch to GitHub:
git push --set-upstream origin backport-1440-to-release-1.9
# Create the PR body template
PR_BODY=$(gh pr view 1440 --json body --template 'Backport 3b31c2e0be030243e4f8ca73a61fcefbb0387ecd from #1440{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title '[release-1.9] allow restricting Kube metadata to local node only' --body-file - --label 'product-approved' --label 'backport' --base release-1.9 --milestone release-1.9 --webOr, if you don't have the GitHub CLI installed (we recommend you install it!): # Push the branch to GitHub:
git push --set-upstream origin backport-1440-to-release-1.9
# Create a pull request where the `base` branch is `release-1.9` and the `compare`/`head` branch is `backport-1440-to-release-1.9`.
# Remove the local backport branch
git switch main
git branch -D backport-1440-to-release-1.9 |
* allow restricting Kube metadata to local node only * integration tests * increase number of cores for test runners * document new option * added extra logging * increased prometheus TSDB retention * restore lower git action machines (cherry picked from commit 3b31c2e)
* allow restricting Kube metadata to local node only * integration tests * increase number of cores for test runners * document new option * added extra logging * increased prometheus TSDB retention * restore lower git action machines (cherry picked from commit 3b31c2e)
Adds the
BEYLA_KUBE_META_RESTRICT_LOCAL_NODEconfiguration option that allows configuring the local informer to only watch the Kubernetes Pods from the local node. This will alleviate the memory load, especially during startup.